Approved  for  public  release;  distribution  is  unlimited 


Dynamic  Database 

Efficiently  convert  massive  quantities  of  sensor  data 
into  actionable  information  for  tactical  commanders 


Mr.  Otto  Kessler 

Program  Manager 

Tactical  Technology  Office 
okessler@darpa.  mil 
703-696-2280 


REPORT  DOCUMENTATION  PAGE 


Form  Approved  0MB  No. 
0704-0188 

Public  reporting  burder  for  this  collection  of  information  is  estibated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and  maintaining  the  data  needed,  and  completing 
and  reviewing  this  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information,  including  suggestions  for  reducing  this  burder  to  Department  of  Defense,  Washington 
Headquarters  Services,  Directorate  for  Information  Operations  and  Reports  (0704-0188),  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington,  VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of 
law,  no  person  shall  be  subject  to  any  penalty  for  failing  to  comply  with  a  collection  of  information  if  it  does  not  display  a  currently  valid  0MB  control  number.  PLEASE  DO  NOT  RETURN  YOUR  EORM  TO  THE  ABOVE  ADDRESS. 


1 .  REPORT  DATE  (DD-MM-YYYY)  2.  REPORT  TYPE  3.  DATES  COVERED  (FROM  -  TO) 

01-06-2000  Conference  Proceedings  xx-xx-2000  to  xx-xx-2000 

4.  TITLE  AND  SUBTITLE 

Dynamic  Database  Efficiently  convert  massive  quantities  of  sensor  data  into  actionable 
information  for  tactical  commanders 

Unclassified 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

Kessler,  Otto  ; 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME  AND  ADDRESS 

DARPA 

Tactical  Technology  Office 
xxxxx,  xxxxxxx 

8.  PERFORMING  ORGANIZATION  REPORT 
NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME  AND  ADDRESS 

Director,  CECOM  RDEC 

Night  Vision  and  Electronic  Sensors  Directorate,  Security  Team 

10221  Burbeck  Rd. 

Ft.  Belvoir,  VA22060-5806 

10.  SPONSOR/MONITOR'S  ACRONYM(S) 

11.  SPONSOR/MONITOR'S  REPORT 

NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

APUBLIC  RELEASE 

13.  SUPPLEMENTARY  NOTES 

See  Also  ADM201258,  2000  MSS  Proceedings  on  CD-ROM,  January  2001 

14.  ABSTRACT 

As  the  number  of  sensors,  platforms,  exploitation  sites,  and  command  and  control  nodes  continues  to  grow  in  response  to  Joint  Vision  2010 
information  dominance  requirements.  Commanders  and  analysts  will  have  an  ever  increasing  need  to  collect  and  process  vast  amounts  of  data 
over  wide  areas  using  a  large  number  of  disparate  sensors  and  information  gathering  sources. 


15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION 

OF  ABSTRACT 
Same  as  Report 
(SAR) 

18. 

NUMBER 
OF  PAGES 
25 

19.  NAME  OF  RESPONSIBLE  PERSON 

Fenster,  Lynn 
lfenster@dtlc.mil 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE 

Unclassified  Unclassified  Unclassified 

19b.  TELEPHONE  NUMBER 

International  Area  Code 

Area  Code  Telephone  Number 

703767-9007 

DSN 

427-9007 

Standard  Form  298  (Rev.  8-98) 
Prescribed  by  ANSI  Std  Z39.18 

Motivation 


response  to  Joint  Vision  2010  information  dominance  requirements,  Commanders  and  anaiysts  will  have  an 
ever  increasing  need  to  coliect  and  process  vast  amounts  of  data  over  wide  areas  using  a  large  number  of 
disparate  sensors  and  information  gathering  sources. 

DARPA’s  Dynamic  Database  (DDB)  Program  is  developing  technoiogy  to  efficientiy  convert  large  volumes  of 
overhead  and  airborne  muiti-sensor  data  into  actionabie  information  for  tactical  commanders.  The  DDB 
challenge  is  to  deai  with  the  problem  of  ground  moving  targets  in  a  brigade  size  area  (nominally  30Km.  X 
30Km.).  Typically  such  an  area  may  contain  1000’s  of  ground  moving  objects  (or  targets).  The  DDB  goai  is  to 
discover  and  maintain  awareness  of  the  location,  kinematics,  tracks  and  identifications  of  such  a  iarge  number 
of  objects  in  such  a  wide  area. 

Data  reievant  to  the  battlespace  are  currentiy  collected  with  a  broad  variety  of  remote  sensors  (SAR,  EO,  IR, 
SIGINT,  GMTI).  These  sensors  are  capabie  of  producing  iarge  numbers  of  reports  in  a  range  of  different 
formats.  The  relevance  of  these  reports  and  the  skills  or  tools  required  to  interpret  them  is  generaily  outside 
the  interest  of  a  tactical  commander.  DDB  will  provide  the  automation  to  to  transiate  relevant  data  from  a 
sensor  perspective  to  a  tacticai  perspective  -  that  is,  a  map  based  view  of  aii  objects  in  the  battlespace. 

Current  processes  requires  sensor  anaiysts  to  rapidiy  sift  through  iarge  volumes  of  data  pertaining  to  wide  areas 
to  assess  friendly  status  and  enemy  situations.  Today’s  anaiysts  are  uniqueiy  trained  with  sensor  specific 
skiiis.  This  stovepiped  process  produces  reports  in  differing  formats  which  require  further  manuai  anaiysis  and 
interpretation  prior  to  use  by  a  force  commander.  The  problem  is  further  complicated  by  the  fact  that  the 
number  of  analysts  is  decreasing  and  that  few  are  trained  to  perform  muiti-sensor  anaiysis. 

The  manual  process  is  unable  to  keep  pace  with  the  time  demands  created  by  the  targeting  decision  cycie 
required  for  responding  to  emerging  threats.  Hostile  ground  forces  will  adopt  new  tactics  that  differ  from 
traditionai  doctrine  and  which  present  difficuit  operating  conditions  for  current  assessment  systems.  Future 
systems  wiil  need  to  expioit  aii  available  sensor  data  more  rapidly  and  with  more  diverse  cues,  in  order  to 
deveiop  and  maintain  a  multi-sensor  view  of  the  the  battiespace,  inciuding  spatiai-temporai  history  of  the 
battiefield  suitabie  for  detecting  tacticaiiy  significant  patterns  and  events.  Such  a  multi-sensor  perspective  will 
be  required  to  support  decision  makers  serving  intelligence  and  operational  functions  at  varying  echeions  of 
command. 


Motivation 


What  the  Commanders  get ... 

-  Large  numbers  of  partially 
overlapping  sensors 

-  100s  of  reports;  1000s  of  images  per 
minute 

-  Unregistered,  soda  straw  sensor 
observations 

-  Very  high  False  Alarm  Rates 

-  Signals  -  based 


■  What  the  commanders  want 

-  Timely  Situation  Knowledge 

-  Comprehensive  Coverage 
(>1000  targets  over  -1000  Km^) 

-  Accurate  target  locations  with 
small  Circular  Error  Probabilities 

-  Low  burden,  geo-referenced 
database 


DDB  Program 


the  battlespace.  Further,  DDB  knowledge  of  objects  in  the  battlespace,  and  the  utility  of  prior  sensor 
collections,  will  provide  the  basis  for  quantified  requests  to  the  collection  management  process  (sensor 
type,  mode,  geometry,  etc.) 

Key  technology  areas  are: 

■  Multi-sensor  Registration 

Registration  of  all  sensor  data  over  space  and  time 

-  Automated  geo-registration  to  a  common  targeting  grid 

-  DIED  /  GIB  /  FFD  reference  data 

-  Object  Oriented  data  structure  and  services 

■  Wide  Area  Change  Detection  (WACD) 

Applied  to  different  sensing  domains  and  leveraging  multi-platform,  multi-aspect,  multi-look  potential,  with 
normalcy  models,  built  on-the-fly,  to  serve  as  a  reference  for  change  recognition. 

-  IMINT 

-  Multiple  parallel  &  independent  (Image  &  Object  Level)  change  detection  paradigms 

-  Dynamic  adaptation  to  scene  context  to  manage  Pd  /  Pfa 

-  Independent  object  detection  and  identification  processes 

-  MTI 

-  Improved  positional  accuracy 

-  Increased  track  continuity 

-  Motion  pattern  /  traffic  flow  change  detection 

-  SIGINT 

-  Reduced  emitter  location  error 

-  New  emitter  mapping  and  profiling  capability 

-  Derivation  of  tactical  communication  networks 

■  All-source  Track  and  Identification  Fusion  (ATiF) 

-  Vehicle  tracks  maintained  through  move  -  stop  -  move  cycles 

-  Aggregates  all  position,  kinematic,  identification  features  for  best  estimate 

-  Recognition  of  sites,  groups,  units,  and  activities 


DDB  Program 


Common  geo-registered  database 

-  Common  grid  tied  to  wide  area  terrain  data  (DIED,  CIB,  FED) 

-  Multi-sensor  observations  (SAR,  EO,  IR,  GMTI  and  SIGINT) 
Fusion  across  sensors 

-  Model  based  evidence  accumulation 


■  Track  targets  and  features  at  object  level 

-  Wide  area  coverage,  large  numbers  of  targets 


■  Dynamic  closed  loop  tasking  overcomes  missing/ambiguous  data 

-  Self  evaluation  of  database  and  sensor  requests 


Dynamic  AAulti- sensor  ISR  Database 


System  Design  and  Technology  Integration 


SDTI  Goal: 


-  Systenn  Design  and  Integration  -  software  architecture,  hardware  architecture  and  laboratory,  and  configuration 
management 

-  GUI  Design  and  Development  -  DARPA  INIMEX-based  zoomable  user  interface 

-  System  /  Fusion  Gain  Modeling 
Key  themes/technologies: 

-  System  Design  and  Technology  Integration 

-  GUI  -  DARPA  INIMEX-based  zoomable  user  interface  to  intuitively  display  sensed  and  derived  fusion  data 
spatially  and  temporally 

-  CESIL  -  (Component  Experimentation  and  System  Integration  Laboratory) 

=  Co-located  Lab  -  High  Performance  computers  and  network  co-located  at  SDTI 
=  Virtual  Lab  -  Networked  via  secure  communications  to  computing  resources  at  co-contractor  sites 

-  System  Modeling 

-  Models  of  sensor  and  algorithm  performance  used  to  analyze  and  predict  performance  of  a  data  fusion  system 

Significance: 

-  System  Design  and  Technology  Integration 

-  Provides  a  distributed  integration  network  at  multiple  security  levels  for  software  development  and  information 
exchange 

-  Provides  platforms  and  environment  to  transition  components  of  DDB  to  demonstration 

-  System  Modeling 

-  Predict  impact  of  different  sensors 

-  Describe  trade  between  better  sensors  and  more  sensing  (Quality,  quantity,  revisit  rate,  etc.) 

-  Evaluate  sensor  mix  issues 

-  Develop  fusion  algorithm  control  methods 

-  Predict  system  performance 


System  Integration  and  Modeling 


System 

Architecture 

•  Component 
Algorithms 

•  High  Performance 
Data  Services 

•  Executive  and 
Control 


Single  INT  Id,  Tracking, 
Change  Detection 


Cross 
INT  Id, 
Tracking, 
Change 
Detection 


Graphical  User 
Interface 


utilizes  the  DARPA  iNiMEX 
zoomabie  interface  concept 

•  Provides  intuitive  dispiay  of 
temporai  and  spatiai  aspects 
of  the  data 

integrates  externai  sensed 
data  and  internai  fusion 
products  on  data  iayers 


(CESIL  -  Component  Experimentation 
and  System  Integration  Laboratory) 


•  Computing  Assets  for  Distributed  Development, 
Experimentation,  and  integration 


AFRL  Virtuai  Distributed  Lab 


hi? 


Government  Labs 


Workstations 

•  UNCLASS  CESIL  WWW, 
Server 

•  Software  Development 


Sun  HPC  6500  SMP 

•  Database  Server  1  TB  RAID  0/3/5 

Runtime  Environment  v 

I 


INTERNET 


DDB  Contractors 


Workstations 

Coilaterai  CESIL  WWW 
Server 

Software  Development 


4.9  TB  DLT 
Tape  Library 


Workstations 

•  Software 
Development 

•  Component 
Testing 


SDTi  CESiL  Components 

Fusion  Performance 
Modeling 

•  Define  sensor  combinations  that 
resuit  in  best  system  performance 

•  Develop  fusion  algorithm  control 
methods 

•  Predict  system  performance 


Sensor 

Operating 

Characteristics 
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Dynamic 
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Model 


DDB  Architecture 


The  High  Performance  Data  Store  (HPDS)  provides  the  dynamic  database  environment  to  rapidiy  store,  index, 
retrieve,  and  share  massive  quantities  of  data  among  DDB  components.  This  data  includes  not  only  sensor  and 
foundation  data,  but  aiso  resuits  of  the  detection,  fusion,  and  reasoning  algorithms.  Therefore,  a  criticai  part  of  the 
DDB  system  is  a  rich  common  object-oriented  schema  that  includes  the  raw  sensor  data,  registration  data, 
features,  associations  such  as  tracks,  and  models  of  real  world  entities  and  their  environment. 

Typical  commercial  DBMSs,  used  as-is,  cannot  meet  DDB's  requirements  to  keep  up  with  the  constant  stream  of 
incoming  sensor  data  and  be  abie  to  index  it  so  that  spatialiy  and  temporaily  coincident  data  can  be  efficientiy 
retrieved.  The  HPDS  is  buiit  using  an  object  database  extended  with  hierarchicai  spatial  and  temporal  indexes  for 
this  purpose.  To  make  room  for  new  data,  obsoiete  data  will  be  efficiently  purged  as  necessary  without 
significantly  impacting  performance. 

Sensor  errors,  registration  errors,  association  errors,  and  possibie  errors  in  identification  aii  contribute  to  uncertainty 
in  DDB  resuits.  We  are  currently  adding  a  rigorous  representation  of  uncertainty  to  the  schema,  and  eventually  in 
the  database  query  engine  itseif,  that  will  allow  algorithms  to  correctly  reason  on  and  combine  uncertain 
information. 

In  addition,  HPDS  provides  active  database  mechanisms  that  can  trigger  an  aigorithm  to  run  on  new  data,  notify  a 
user  of  a  significant  event,  or  determine  what  computations  must  be  run  to  answer  a  user's  ad  hoc  query.  The 
system  can  be  “loosely”  distributed  across  multipie  locations,  with  data  and  queries  seiectiveiy  repiicated  to 
support  an  operational  environment  with  possibly  unreliable  communications  networks. 


DDB  Architecture 

Enables  Sensor  Data  Access  and  Technology  Growth 


Registra 


k 


•  Scalable  Database  architecture  overcomes  limitations  of  data  ownership; 
Enhances: 

-  Data  sharing,  Interoperability 

-  Technological  growth  -  applications  and  visualization 
Leverages  COTS  thrusts  in  ‘open  systems’  &  ‘object-oriented  databases’ 

-  Explores  military  needs  which  exceed  likely  commercial  interests 

User  Interface 

standing  Queries  & 
AutomaJj^Notification 
Info 

Products 


High  Performance  Data  Store 

00  schema  •  Rapid  search,  retrieval,  purging  •  Rapid  space-time  queries  • 
Probabilistic  queries  •  Query  &  data  distribution  •  Database  mediation  •  Selective  data  push 


Scalable  to  Massive,  Dynamic  Datasets 
in  Operational  Environments 

Situation  History  Database 


Change  Detectors,  All  source  Track  Fusions 
Model  Based  Classifier,  Registration,  .. 


Applications 


Registration 


■  Registration  Goai: 

The  DDB  Registration  process  of  geospatial  normalization  supports  accurate  and  efficient  change  detection  and  fusion 
processes.  Muiti-INT  (GMTI,  SIGINT,  IMINT)  data  is  reoriented  to  a  common  targeting  grid  during  ingest 

-  Giobally-consistent  multi-temporal  observations  share  common  map  coordinates 

-  position  accuracy  increases  over  time  through  compiementary  sensor  observations 

■  Key  themes  /  technoiogies: 

-  aii  DDB  objects  geolocated  in  3-D  (latitude,  longitude,  elevation) 

-  Cross  platform  systematic  errors  are  autonomously  removed  or  reduced  during  “background”  processing 

-  Registration  metrics  provide  operator  insight  into  DDB  registration  accuracy 

-  Registration  success  metric 

-  CE90/LE90  error  statistics 

-  Position  knowledge  of  DDB  objects  and  reference  data  continuously  refined  and  improved  over  time 


Cross-INT  salient  features  automaticaiiy  defined  and  refined  as  Precision  Georegistered  Invariant 
Features  (PGIF) 

PGIFs  are  features  that  occur  across  Muiti-INT  data  (i.e.  road  intersections,  stationary  rotators,...) 
PGIFs  are  used  to  make  associations  in  Muiti-INT  data  for  error  propagation  and  correction 


■  Significance: 

Geospatiai  Normalization  Enables  Change  Detection  and  Fusion 

-  Performance  of  OLCD  is  significantiy  improved  (higher  detection  rate,  iower  false  alarm  rate) 

-  Performance  of  GMTI  tracking  algorithms  is  improved  (fewer  track  losses,  higher  fidelity  tracks) 

-  Abiiity  to  support  multi-INT  wide  area  change  detection 

-  Abiiity  to  reduce  false  detections  and  false  associations 


Registration  of  aii  sensors  to  a 
Common  Targeting  Grid 


DDB  APPLICATIONS 


Normalcy  Models 


■  Nomalcy  Modeling  Goal: 


-  Normalcy  Modeling  for  Change  Detection 

-  Multispectral  Image  Intelligence  (IMINT)  for  Targets  and  Scenes 
Emissions  Patterns  (SIGINT) 

-  Traffic  Patterns  (GMTI) 


■  Key  themes  /  technologies  : 

-  High-Fidelity  Off-line  Modeling 

-  Rapid  On-line  Target  and  Scene  Modeling  for  On-line  Reasoning 

-  Target  Configurations,  Articulations,  and  Extended  Operating  Conditions 

-  Scene  Geospatial  and  Statisitical  Clutter  Variations 

-  Sensor  Acquisition  Geometry  Modeled  On-line 

-  Multispectral  Segmentation  of  IMINT  Data  to  Improve  Context 

-  Multispectral  Validation  of  Technology  Against  Multiple  Sensors 

-  On-line  Geospatial  and  Temporal  Learning  to  Update  Normalcy  Models  and  3-D  Site/Region  Context 

■  Significance: 

-  Integrated  Multispectral  Context  Based  Change  Detection 

-  Scene  Context  from  IMINT 

Behavioral  Context  from  GMTI  and  SIGINT 

-  Ability  to  Separate  Significant  Changes  from  Normal  Variance  Using  Multispectral  Sensors 

-  Exploit  Strengths  of  Each  Sensor  through  Data  Fusion  at  Reasoning  Levels 

-  Rapid  3-D  Context  Generation  and  Updates  for  Sites  and  Regions  of  Interest 

-  Multispectral  High-Fidelity  On-line  Target  Signature  Generation  for  Target  Reasoning 


Normalcy  Models  enable 
Wide  Area  Change  Detection 
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IMINT,  GMTI,  SIGINT 
Sensing  parameters,  Terrain, 
Landclass,  Object  relationships 


Normalcy  models  provide  context 
for  detection  thresholds 

m  Is  this  a  region  of  high  clutter? 

■  Was  it  there  on  the  last  sensing  pass? 

■  Has  it  changed  state  or  shape? 

■  Is  it  emitting  as  expected? 

■  Is  it  moving  in  a  new  way,  place,  or  time? 

Combined  multi-sensor  data  Is 
required  to  derive  normalcy 

m  Spatial,  Temporal,  Feature  based 
representation  of  scene  content, 
background,  and  behavior. 


Feedback  from 
Track  and  ID 
Fusion 


SIGINT  Change  Detection  Approach 


-  Using  data  from  multiple  platforms  provide  a  significant  increase  in  COMINT  emitter  geolocation  accuracy  and  a 
significant  increase  in  tracking  performance. 

-  Develop  alternate  methods  of  automated  evaluation  of  SIGINT  data  that  can  be  used  to  find  tactically  significant  events. 

■  Key  themes  /  technologies 

-  Multi-Platform  Track  and  ID: 

-  Data  on  each  emitter  from  multiple  sources  fused  into  single  report 

-  Platform  type  recognition 

-  Tactical  alerts 

-  Emission  Density:  Determine  distribution  of  significant  RF  activity  across  the  battle  field 

-  Emission  Profile:  Derive  modes  of  activity  based  on  temporal  communication  patterns  and  determine  normal  and 
abnormal  activity. 

-  Communication  Networks:  Find  and  identify  the  nodes  and  links  of  operational  communication  networks 

■  Significance 

-  Multi-Platform  Track  and  ID: 

-  Improved  geolocational  accuracy 

-  Greatly  improved  ID  and  vehicle  recognition 

-  Ability  to  extract  intent  and  alerts 

-  Emission  Density:  Aids  in  the  location  and  ID  of  militarily  significant  sites  and  trends 

-  Emission  Profile:  ID  of  tactically  significant  events  and  activities. 

-  Communication  Networks:  Identifies  command  centers  and  command  networks 


SIGINT  Change  Detection 

Approach 


SIGINT 

INGEST 


Multi-Platform 
Emitter  Correlation 


►SIGINT  Ingest: 
•Ingest  data  from 
multiple  systems 
into  the  HPDS 

Automated 

ID 


Multi-Platform  Emitter 
Tracking 

. <5) 
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Emission  Density 

■  A 


Comm  Networks 


Change  Detection/ 
Event  Recognition 


►Automated  ID: 
•Use  results  from 
AFRL  (Rome)  work 
to  supply  Speaker 
ID,  Vehicle  ID  and 
Internal  ID 


Multi-Platform 
Correlation/Fusion 

►Geolocation  Correlation  and  ID 

•Multi-Platform  Emitter  Correlation 

-  correlates  reports  based  on 
geolocation  and  ID. 

•Multi-Platform  Emitter  Tracking  - 

correlates  emissions  over  time  based  on 
ID  parameters 


Coarse  Reasoning 

•Analysis  of  spatio-temporal  structure  of  RF 
activity: 

•Emission  Density  -  exploits  spatial  features  of  the 
battlefield 

•Emission  Profile  -  exploits  periodic  activity  of  the  battle 
space 

•Communication  Network  -  maps  lines  of  communication 
between  entities  and  echelon  levels. 


■  GMTI  Change  Detection  Goal: 

GMTI  change  detection  will  improve  the  ability  to  detect  and  track  dynamic  ground  forces.  GMTI  Sensor  History 
Database  takes  advantages  of  single  and  multiple  GMTI  platforms  (different  viewing  angles)  to  increase  track 
continuity  of  high  vaiue  targets,  and  provide  early  indications  of  significant  movement  change. 

■  Key  themes  /  technologies: 

-  MTI  Ingestor  -  Ingests  GMTI  from  multiple  sources:  Joint  STARS  MC44,  AFRL  Normaiized  MTI  Format 
(NMTI),  etc.  to  construct  persistent  common  GMTI  database  objects  representing  the  measurements  and 
sensor  coliection  metadata 

-  Absoiute  MTI  Registrar  (AMTIR)  -  GMTI  sensor  independent  debiasing  of  GMTI  measurements. 
Registration  reiative  to  (registered)  road  network 

-  Muitipie  Piatform  GMTI  Tracking  -  Fusion  o  f  GMTI  from  multiple  sensors.  Uses  terrain  context:  roads, 
terrain  mobiiity  constraints 

-  Trip  Wire  Sentry  (TWS)  and  Flow  Change  Detection  -  Measures  movement  of  traffic  fiow.  Statistical 
Modeling  of  normal  flow  and  detection  of  significant  change  in  flow 

-  MTI  Pattern  Change  Detection  -  Detect  new  active  Lines  of  Communications,  active  sites  and  spatiai- 
temporal  features  of  GMTI  linear  and  isoiated  ciusters 

■  Significance 

-  Provides  rapid  retrievai  of  common  representation  of  GMTI  using  spatial-temporai  collections 

-  Improves  multi-source  fusion  via  improved  position  accuracy 

-  Improves  track  continuity,  purity,  position  and  velocity  accuracy 

-  Provides  indications  and  warning  of  significant  movement 

-  Detect  active  sites,  potential  Lines  of  Communications,  barracks,  motor  poois,  construction 
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Wide  area  movement  estimates 
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Multiple  Platform  MPCD  -  MTI  Pattern  Change  Det.  TWS  -  Trip  Wire  Sentry  FlowCD  -  Flow  Change  Det. 


■MINT  Change  Detection  Goals: 

The  DDB  IMINT  Change  Detection  Approach  exploits  multi-sensor,  multi-look,  and  multi-temporal  imagery  using  a  unique 
combination  of  image-level  and  object-level  change  detection  algorithms  to  provide  real-time  recognition  of  object  presence  and 
extraction  of  ID  features. 


These  complimentary  algorithmic  approaches  tend  to  be  orthogonal  enabling  decorrelation  of  false  alarms.  This  enables  the  IMINT 
Change  Detection  algorithms  to  run  at  higher  sensitivity  levels  with  greatly  reduced  false  alarms.  This  innovative  approach  to 
change  detection  produces  a  semantically  richer  set  of  features  that  are  used  for  high  confidence  discovery  of  militarily  significant 
object  changes  for  simultaneous  tracking  and  target  identification. 

This  approach  overcomes  current  state-of-the-art  change  detection  approaches  which  do  not  perform  robustly  under  extended 
operating  conditions  (e.g.,  scene  diversity,  target  adjacency,  and  partial  obscuration),  and  which  do  not  manage  false  alarms  well 
when  operating  at  high  sensitivity  levels  (I.e.,  high  Pd)  or  when  subject  to  nuisance  parameters  like  aspect  diversity  and  cardinal 
angle  flash. 


Key  themes  /  technologies: 

-  SAR  Object-Level  Change  Detection  (SAR  OLCD) 

-  EO  Object-Level  Change  Detection  (EO  OLCD) 

-  SAR/EO  Image-Level  Change  Detection  (SAR/EO  ILCD) 

-  Object-Level  Change  Fusion  (OLCF) 


V. 


ILCD  performs  temporal  segmentation  followed  by  spatial  anomaly 
detection  which  compliments  OLCDs  spatial  segmentation  followed 
by  temporal  anomaly  detection.  The  use  of  normalcy  models  as  well 
as  context  to  increase  Pd/Pfa  is  used  by  all  three  change  detection 
approaches. 


Significance: 


-  Effective  image  screening  and  cueing  -  vast  data  reduction  of  multi-sensor  imagery  to  ID  of  militarily  significant  entities 


-  Fusion  gain  cross-sensor,  cross-algorithm,  across  extended  operating  conditions 


-  Maximum  use  of  all  available  multi-sensor  imagery: 

=  Maintain  performance  over  wide  areas  using  low  quality  data  (e.g.  Stripmap,  search  mode) 

=  Extend  performance  over  focused  areas  through  use  of  limited  high  quality  data  (e.g.  Spotlight) 


-  Dynamic  adaptation  to  scene  context  through  use  of  “Normalcy”  models 


=  Background  adaptation 


=  Change  detection 
=  Target  identification  and  state 


■MINT  Change  Detection 
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■  All  Source  Track  and  Identity  Fusion  (ATIF)  Goals: 


The  All  Source  Track  and  Identity  Fusion  (ATIF)  component  takes  as  input  MTI,  IMINT,  and  SIGINT  tracks.  It 
produces  as  output  all  source  tracks  by  associating  the  input  tracks,  and  fusing  the  corresponding  kinematic 
and  attribute  state  information.  ATIF  is  currently  being  tested  on  simulated  input  tracks,  as  well  as  tracks 
derived  from  JSTARS  MTI  and  several  sources  of  SAR  and  EO  IMINT  data. 

From  a  tracking  perspective,  ATIF  has  two  primary  benefits.  The  first  is  that  it  improves  track  continuity  -  it 
enables  the  warfighter  to  maintain  target  track  longer  because  ATIF  can  track  through  move-stop-move 
cycles  by  associating  MTI  and  IMINT  hits.  Maintaining  target  track  longer  simplifies  situation  estimation 
because  analysts  have  fewer  track  fragments  to  interpret.  Current  testing  indicates  that  track  continuity  can 
be  extended  by  approximately  a  factor  of  3-4. 

The  second  primary  benefit  of  ATIF  is  that  it  improves  state  estimates.  Target  classifications  are  improved  via 
fusion  of  classification  information  across  INTs  and  over  time  -  the  greatest  impact  of  this  is  on  MTI-based 
mover  tracks  which  can  be  provided  with  target  classification  information  via  association  to  SIGINT  or  IMINT 
tracks  that  are  far  easier  to  classify.  Kinematic  state  estimates  are  also  improved  via  fusion  of  kinematic 
information  across  INTs  and  over  time  -  the  greatest  impact  of  this  is  on  tracks  derived  from  SIGINT  that  can 
be  associated  to  MTI  and  IMINT  tracks,  and  on  MTI  track  starts/ends  that  can  be  associated  to  IMINT  hits. 

■  Key  themes  /  technologies 

ATIF  is  based  on  multiple  hypothesis  tracking  (MHT)  technology.  The  MHT  approach  uses  statistical  models  of 
target  state  dynamics  across  move-stop-move  cycles  and  uncertainties  in  sensor  kinematic  and  attribute 
information.  The  practical  application  of  MHT  technology  relies  on  optimization-based  hypothesis 
management  methods  to  maintain  and  score  multiple  hypotheses,  while  scaling  to  realistic  problem  sizes. 
ATIF  currently  runs  in  greater  than  10x  real  time  on  scenarios  with  6OO-1-  targets  on  a  standard  PC 
configuration  and  operationally  realistic  sensor  sampling  rates  (e.g.,  ~10  secs  MTI  sampling  period). 

ATIF  has  a  generic  sensor  interface  that  allows  it  to  be  straightforwardly  upgraded  to  accept  tracks  from 
multiple  sensors  of  the  same  type  (e.g.,  multiple  SIGINT  correlators),  to  fuse  new  attributes  (e.g.,  geometric 
attributes),  and  to  process  data  from  new  sensor  types  (e.g.,  unattended  ground  sensors). 

■  Significance: 

ATIF  technology  will  enable  much  more  effective  exploitation  of  multi-INT  sensor  data  for  situational  awareness 
and  target  planning 

The  capability  to  maintain  track  through  move-stop-move  cycles  will  ease  burden  on  analysts  and  operators 
via  extension  of  track  continuity. 
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Fuses  Change  Detector  outputs 

-Detections,  Tracks,  Identifications,  Features 
-MHT  (Multiple  Hypothesis  Tracker) 

-  Support  sensor  tasking 

Target  Identification 

-Fuse  and  Maintain  SIGINT/  IMINT  Identity  Information 

-  Model  Based  Classifier 

-  EO  and  SAR  classification  of  vehicles 
-SIGINT  ID  features  link  with  moving  and  stationary  cycles 

Target  Track  Continuity  through 
move-stop-move  cycles 

-Fuse  GMTI  and  SIGINT  when  moving 
-Fuse  IMINT  and  SIGINT  when  stopped 

-  Increase  Track  Length  Relative  to  MTI  Alone 

-  Recognize  force  level  activity  and  relationships 
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Force  Level  Change  Detection  (FLCD) 


FLCD  Goals: 

The  goal  of  this  experimental  thread  is  to  estabiish  a  foundation  for  automated  reasoning  about  force  levei  groups 
and  activities  as  supported  by  evidence  from  muitiple  platform,  multi-INT  fused  products  in  the  dynamically 
updated  data  base. 

The  approach  introduces  new  technoiogies  focused  on  bridging  the  gap  between  bottom  up  evidence  based 
deterministic  fusion  and  top  down  probabiiistic  reasoning.  Tacticai  bottom  up  fusion  is  typically  used  to  support 
IPB,  l&W,  and  ATR.  It  answers  the  question,  “What  does  the  evidence  mean?”.  Top  down  reasoning  is  used 
tactically  to  make  decisions  about  taking  action  over  expected  threat  and  targeting  opportunities.  It  typically  asks 
the  question,  “Does  the  evidence  support  my  beiief  about  what  is  happening?” 

Objectives  that  support  these  goais  include: 

-  Designing  an  application  framework  and  engine  to  identify  sites,  groups,  units  and  determine  their 
associated  activities 

-  Track  state  changes  of  these  force  levei  activities  across  space  and  time 

-  Characterize  and  assess  the  confidence  ievei  of  reported  conciusions 

Key  themes  /  technologies: 

The  key  hypothesis  is  that  automated  force  level  reasoning  is  feasible  and  can  yield  meaningful  force  level 
change  detection.  The  DDB  component  deveioped  to  test  this  hypothesis  is  called  TSGUDA  (Tactical  Site, 
Group,  and  Unit,  Detection  and  Assessment).  TSGUDA  technoiogy  combines  other  DDB  fused  products  and 
coarse  data,  including  ATIF  tracks  SIGINT  emission  densities,  and  MTI  flow  densities  with  weather,  terrain  and 
feature  data,  and  current  situation  information  using: 

Bayes-net  and  frame  system  knowiedge  representation 
Model  Fragments  specified  by  domain  subject  matter  experts 

-  On  the  fly  hypothesis  generation  with  muitipie  hypothesis  management  and  control  algorithms 

-  The  fidelity  of  force  ievei  hypotheses  are  measured  by  situation  metrics  that  include  P(D)  and  P(FA)  for 
groups 

Significance: 

Successful  detection  and  tracking  of  site/unit/group  activities  ( i.e.  at  acceptabie  confidence  ievels)  wiii  : 

-  Estabiish  the  foundation  for  providing  interactive  user  control  over  the  fusion  process 

-  Provide  context  required  to  make  COA  decisions  over  tactical  targeting  activities 

-  Provide  the  basis  for  prioritizing  sensor  tasking  and  coiiection  requests 


Force  Level  Change  Detection 


Information  to  Support 
Control: 

Context:  current  target 
assessments 

•  Focus:  tasking  priorities 


^  TSGUDA  :  Tactical  Site,  Group,  and  Unit,  Detection  and  Assessment  ^ 


DDB  Algorithm  Relationship 


The  DDB  Algorithm  Relationship  chart  depicts  notional  data  flow  through  the  algorithms  and  is  not  intended  as  a 
“wiring  diagram”.  The  algorithms  interface  with  sensor  data  and  derived  products  which  reside  in  an  object  oriented 
database.  An  ongoing  system  integration  and  system  modeling  effort  identifies  integration  issues  and  bounds  fused 
system  performance  expectations. 

MTI,  IMINT  and  SIGINT  data  are  co-registered  to  terrain  information  (DTED,  GIB,  FFD).  Invariant  features  are  used 
as  match  points,  for  example,  a  stationary  rotator  that  emits  can  be  detected  by  MTI  as  it  sweeps,  “seen”  by  IMINT 
and  “heard”  by  SIGINT.  High  resolution  IMINT  is  used  to  refine  terrain  information  for  the  next  registration  cycle. 

Co-registered  multi-sensor  data  is  used  to  rapidly  create  Normalcy  Models.  Change  Detectors  use  the  normalcy 
models  as  the  reference  to  process  subsequent  ingested  data.  Image  Level  and  Object  Level  Change  Detectors 
(ILCD  &  OLCD)  process  IMINT  data.  The  Object  Level  Change  Fusion  (OLCF)  process  fuses  the  ILCD  and  OLCD 
outputs  which  de-correlates  false  alarms  and  increases  probability  of  detection.  OLCF  and  the  MTI  and  SIGINT 
change  detectors  provide  track  data  to  the  All-source  Track  and  ID  Fusion  (ATIF).  OLCF  cues  the  Model  Based 
Classifier  (MBC).  The  MTI  and  SIGINT  change  detectors  also  provide  flow  density  and  emissions  density, 
respectively,  to  the  Force  Level  Change  Detector  (FLCD). 

ATIF,  including  the  FLCD  function,  fuses  all  the  processed  data  into  an  integrated  track  file  that,  for  the  first  time  ever, 
tracks  ground  objects  through  their  “move-stop-move”  cycles  and  identifies  the  objects.  ATIF  maintains  multiple 
hypotheses  regarding  association  of  data  to  objects  and  tracks,  and  uses  succeeding  data  to  resolve  ambiguities 
and  improve  track  estimation  quality.  FLCD  uses  the  output  of  the  ATIF  process  combined  with  flow  and  emissions 
density  to  identify  activity  related  to  sites,  groups  and  units. 
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Object  Oriented  Database  Management  System 


■  Registration: 

-  Shared  multi-spectral 
fiducial 

-  Salient  feature  matching 

-  Iterative  refinement 

■  Change  Detection: 

-  Adaptive  background 
modeling  (normalcy) 

-  Spatial  and  temporal  pattern 
analysis 

-  Tripwires 

■  All-Source  Track  &  ID: 

-  Correlates  detection 
features  from  sensors 

-  Accumulate  over  time 

-  Maintain  through 
move/stop/move  cycles 

1  LCD:  Image  Level  Change  Detection  ^  OLCD:  Object  Level  Change  Detection  ^  OLCF:  Object  Level  Change  Fusion  ^  ATIF:  All-Source  Track  &  ID  Fusion 

